This section is intended to introduce the reader to various aspects of art, which may be related to various aspects of the present principles that are described and/or claimed below. This discussion is believed to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the present principles. Accordingly, it should be understood that these statements are to be read in this light, and not as admissions of prior art.
Today films are mostly produced for a central screen only. To improve user experience, additional content is projected around the television or cinema screen thus extending the original scene around the user. Such system by filling the viewers' field of vision enhances immersiveness of the user, i.e. it gives the user a better sense of being in the scene. Such content is for instance captured using wide field of view cameras, rig of cameras for 110 to 360 capture, or created using computer generated images.
However extending video content in a manner that is consistent with the original video and visually pleasing is difficult.
Among methods for extending video content, the first one is to capture additional content with a wide field of view. However such technique requires a lot of manual work at least because of the post-production need to manage more images (central and side images). Besides special care need to be taken for the extension which should not disturb nor create troubles for the viewer as it is designed to improve user experience.
A simple technology for extending video content is the technology Ambilight by Philips, disclosed in WO 2004/006570, which studies the distribution of colors in the central image and projects main colors of the image around the screen using RGB LEDs mounted on the edges of the TV. The light projection is only done around the TV in a limited area and provides no detail of background nor shape nor motion impression. However such extensions coherent with content in terms of color and light might not be optimal in case of high luminance image because such technology increases too much the ambient light. The extension should not disturb nor create troubles for the viewer as it is designed to improve user experience.
Another technology for extending video is the technique disclosed by A Aides et al. in “Multiscale ultrawide foveated video extrapolation” (in IEEE International Conference on Computational Photography (ICCP) 2011, pages 1-8). The key idea of the algorithm is to use a Patch-Match algorithm in space and time dimensions (using video cubes, instead of image blocks) to extrapolate the video outside of its borders. When choosing the right patch, the method penalizes patches that are likely to cause distraction. Besides, since human vision is less sensitive to details when it gets farther from the center of the eye, the algorithm is computed at different resolution. The best results are obtained when the extrapolation is done from the coarser resolution to the highest resolution. However with such techniques, the whole field of vision of a user may not be filled since the size of the extensions is limited. Besides, as for other computer generated images, the artistic intent might not be preserved. Indeed, the user might be distracted from the narrative by side images or modification of the viewing conditions that may be too attractive.
Thus even if extensions are desirable for immersiveness, extensions may not be suitable for a comfortable viewing experience, in particular when too much light is induced by the extended projections. A method for processing extended content so that user experience is improved is therefore needed.